Repository navigation
Read remote files: client.file(path) with byte ranges, streaming and digests (#146) - #177
Conversation
…digests (#146) Foundation of the remote file access family (#146, then #145, #147, #148): client.file(path) returns a RemoteFile request with readBytes(), readText(Charset), openStream(), openReader(Charset) and digest(String), plus offset(long) (negative = from the end), length(long), maxBytes(long) (64 MiB default cap) and timeout(Duration). The package-private RemoteFiles primitive runs a small PowerShell script (-EncodedCommand, path embedded base64-encoded, no quoting) that opens the file with a ReadWrite share mode, seeks, and writes each block as one base64 line to the raw stdout stream; the client decodes the lines as they arrive and maps the script's exit code to explicit failures (not found, directory, access denied, sharing violation, Constrained Language Mode, PowerShell missing). Blocks are 49,149 bytes so each 65,533-byte line fills exactly two of the WinRM service's 32 KiB pipe reads (at most ~60 per second per stream): 20 MiB now reads in 14-15 s on Windows Server 2008 R2, 2019 and 2022, instead of 101 s with Console.Out-written 76 KB lines. Tests: the scripts run in the local powershell.exe on Windows (RemoteFilesScriptTest), client-side decoding against FakeWsmanServer (RemoteFileTest), and live tests; docs in the new files.md page. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc9693dbe5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ADME - readBytes(): a read that fills exactly the requested length now drains the stream to its end, so the script's exit code is still checked and a late failure is not reported as a successful range read. - A path too long for the reader script to fit the remote command line (about 1,450 ASCII characters) is refused with a WinRMClientException before anything is sent, instead of silently falling back to uploading the script: a read never writes on the host. Documented in files.md. - readText()/openReader() Javadoc: nothing is stripped, a byte order mark is handled as the given charset's decoder handles it (UTF-16 consumes it). - README: revert the quick-start addition (AGENTS.md keeps README terse; the feature is documented in files.md). Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
@codex please review again |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Closes #146 — first issue of the remote file access family (#146 → #145 listing, #147 download, #148 CLI).
What
WinRMClient.file(path)→RemoteFilerequest:offset(long)(negative = from the end, resolved on the host from the open stream),length(long),maxBytes(long)(64 MiB default cap for the in-memory terminals, also sent to the host),timeout(Duration).RemoteFilesprimitive: a PowerShell script (-EncodedCommand, path embedded base64-encoded: no quoting) opens the file with aReadWriteshare mode (logs held open by a service can be read), seeks, and writes each block as one base64 line to the raw stdout stream. The client decodes lines as they arrive; the exit code, checked at the end of the output, becomes an explicitWinRMClientException(not found, directory, access denied, sharing violation, Constrained Language Mode, PowerShell missing). A missing file fails inopenStream()itself; a failure midway is never a silently short read.files.mdpage (menu, index, cross-link fromfile-transfers.md), README one-liner.Performance
Investigated in depth: the WinRM service reads a command's stdout pipe itself, at most 32 KiB per read, ~60 reads/s per stream — not CPU (host 97–99 % idle; client ingests 24–31 MB/s). Blocks are therefore 49,149 bytes, so each 65,533-byte line fills exactly two reads. Measured with a 20 MiB file, SHA-256 verified:
Follow-ups opened from these measurements: #175 (raw bytes instead of base64, ~+33 %) and #176 (parallel ranged downloads, ~3× with 4 connections).
Tests
RemoteFilesScriptTestruns the generated scripts in the localpowershell.exe(Windows only, skipped elsewhere): every byte value across blocks, non-ASCII name, ranges, negative offsets, a file held open by a writer, a locked file → sharing violation, exit codes, digest.RemoteFileTest(FakeWsmanServer): base64 lines split across Receive chunks, a UTF-8 character split across blocks, BOM kept, size cap sent and enforced, failures, unexpected output, wall-clock timeout.WinRMLiveTest: byte-exact read of a non-ASCII-named file, tail range, digest; 1 GiB tail served by a seek; a file held open for writing by another session. Passed on anaxagore (2008 R2) and tc-win2022.mvn clean verify sitegreen: no checkstyle/PMD/SpotBugs findings.Deviations from the issue
readText(Charset)/openReader(Charset)take the charset as a parameter (nocharset()setter, no default to forget).digest()hashes with .NET on the host, reusing the same script and exit-code mapping, rather thanShellFileCopy'scertutilprobe (which goes throughcmd.exequoting).🤖 Generated with Claude Code